Displaying DCI and other content on an enhanced dynamic range projector

ABSTRACT

Systems and methods of rendering DCI-compliant image data on Enhanced Dynamic Range (EDR) display systems are disclosed. One embodiment of an EDR projector system comprises a first modulator and a second modulator. One method for rendering DCI-compliant image data on an EDR projector system comprises: receiving input image data, said image data comprising a plurality of image formats; determining whether the input image data comprises DCI image data; if the input image data comprises DCI image data, then performing dynamic range (DR) processing on the DCI image data; and rendering the dynamic range processed DCI image data on the EDR projector system. One DR processing method is to set the first modulator to a desired luminance level—e.g., fully ON or a ratio of DCI max luminance to the EDR max luminance. In addition, a desired minimum level of luminance may be set for the EDR projector.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/449,647 filed on Jun. 24, 2019, which is a continuation of U.S.patent application Ser. No. 15/025,536, filed on Mar. 28, 2016, (nowU.S. Pat. No. 10,368,044), which is the 371 national stage of PCTApplication No. PCT/US2014/058905, filed Oct. 2, 2014, which in turnclaims priority to U.S. Provisional Patent Application No. 61/889,322,filed on Oct. 10, 2013, each of which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The present invention relates to displays systems and, moreparticularly, to Enhanced Dynamic Range (EDR) projection displaysystems.

BACKGROUND

The Digital Cinema Initiative (DCI) is a joint venture of major motionpicture studios, started in March 2003. DCI's primary purpose was todevelop and promote specifications for an emerging digital cinemamarket. These specifications include a set of content requirements toenable compatibility and interoperability.

Among the various parts of the “Digital Cinema System Specification”first released on Jul. 20, 2005 (and subsequently updated periodically),there is a set of performance specifications for the projection systemsand/or its content regarding color space, resolution, brightness,contrast and interfaces.

As the technology of projector systems progress, however, theperformance of today's most advanced projectors may well outperform theprojector systems as specified in the DCI specifications.

SUMMARY

The present invention provides for the preparation and display contenton advanced high performance projectors and other displays. Suchdisplays include various embodiments of large format, wide color gamut,and high dynamic range cinema projectors and other displays. Suchprojectors are desirably utilized in all cinema applications includingtraditional theater movie houses and multiplexes, television, closedcircuit applications, live concerts, sporting events, theme parks,billboard advertising, and professional applications of all typesincluding industrial design systems, cinema post-production, graphicarts, publishing, etc. Such displays typically have a contrast ratiothat exceeds typical cinema standards or modern displays includingcontrast ratios of more than 5,000 to 1 and may be 1,000,000 to 1 andhigher in some circumstances. Such displays typically have a color gamutthat exceeds current cinema standards. Providing and displaying contentfor several embodiments of display systems and methods of theirmanufacture and use are herein disclosed.

Systems and methods of rendering DCI-compliant image data on EnhancedDynamic Range (EDR) and other high performance display systems aredisclosed. One embodiment of an EDR projector system comprises a firstmodulator and a second modulator. Other embodiments may provideincreased resolution modulators with any of laser light sources, contentenhancements including resolution and/or frequency domain processing andprojection of modified sub-frames in parallel (e.g., multipleprojectors) or alternating manners (in single or multiple projectormodes).

In one embodiment, a method for rendering DCI-compliant image data on anEDR projector system comprises: receiving input image data, said imagedata comprising a plurality of image formats; determining whether theinput image data comprises DCI image data; if the input image datacomprises DCI image data, then performing dynamic range (DR) processingon the DCI image data; and rendering the dynamic range processed DCIimage data on the EDR projector system. One DR processing method is toset the first modulator to a desired luminance level—e.g., fully ON or aratio of DCI max luminance to the EDR max luminance. In addition, adesired minimum level of luminance may be set for the EDR projector.

In another embodiment, a multi-modulation projector display systemcomprises: a light source; a controller; a first modulator, said firstmodulator being illuminated by said light source and said firstmodulator comprising a plurality of analog mirrors to modulate lightfrom the light source; a second modulator, said second modulator beingilluminated by light from said first modulator and capable of modulatinglight from said first modulator, and said second modulator comprising aplurality of mirrors; said controller further comprising: a processor; amemory, said memory associated with said processor and said memoryfurther comprising processor-readable instructions, such that when saidprocessor reads the processor-readable instructions, causes theprocessor to perform the following instructions: receiving input imagedata, said image data comprising at least one highlight feature;determining whether the input image data comprises DCI image data; ifthe input image data comprises DCI image data, performing dynamic rangeprocessing on the DCI image data; and rendering the dynamic rangeprocessed DCI image data on the EDR projector system.

Other features and advantages of the present system are presented belowin the Detailed Description when read in connection with the drawingspresented within this application.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments are illustrated in referenced figures of thedrawings. It is intended that the embodiments and figures disclosedherein are to be considered illustrative rather than restrictive.

FIG. 1 depicts one embodiment of an EDR projector display system thatmay be suitable for the systems, methods and techniques of the presentapplication disclosed herein.

FIG. 2 depicts another embodiment of an EDR projector display systemthat may be suitable for the systems, methods and techniques of thepresent application disclosed herein.

FIG. 3 depicts one embodiment of a high level diagram of the opticalprocessing/image processing that may be affected by the dual,multi-modulator display system.

FIG. 4 depicts one embodiment of a method for producing a suitablebinary halftone image.

FIG. 5 depicts one embodiment of a technique for generating apulse-width DMD compensation image.

FIG. 6 is one embodiment of a flow diagram of using premod-to-primarymaps, light field models and primary-to-premod maps to produce a primaryregistered light field.

FIG. 7 is one embodiment of a DCI-mode image processing pipeline as madein accordance with the principles of the present application.

FIG. 8 is one embodiment of a dynamic range processing module as made inaccordance with the principles of the present application.

FIG. 9 is another embodiment of a dynamic range processing module asmade in accordance with the principles of the present application.

FIG. 10 is a representation of a color gamut map of an EDR projectorsystem as compared with DCI P3 color gamut map.

DETAILED DESCRIPTION

Throughout the following description, specific details are set forth inorder to provide a more thorough understanding to persons skilled in theart. However, well known elements may not have been shown or describedin detail to avoid unnecessarily obscuring the disclosure. Accordingly,the description and drawings are to be regarded in an illustrative,rather than a restrictive, sense.

As utilized herein, terms “component,” “system,” “interface,”“controller” and the like are intended to refer to a computer-relatedentity, either hardware, software (e.g., in execution), and/or firmware.For example, any of these terms can be a process running on a processor,a processor, an object, an executable, a program, and/or a computer. Byway of illustration, both an application running on a server and theserver can be a component and/or controller. One or morecomponents/controllers can reside within a process and acomponent/controller can be localized on one computer and/or distributedbetween two or more computers.

The claimed subject matter is described with reference to the drawings,wherein like reference numerals are used to refer to like elementsthroughout. In the following description, for purposes of explanation,numerous specific details are set forth in order to provide a thoroughunderstanding of the subject innovation. It may be evident, however,that the claimed subject matter may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to facilitate describing the subjectinnovation.

Introduction

Enhanced Dynamic Range (EDR) projection systems have significantlyimproved performance as compared to a standard DCI (Digital CinemaInitiative) projector. Display of standard DCI content is designed andgraded for projectors that meet the DCI projector specification. Theseprojectors have a more limited dynamic range and color space than theEDR projector. In order to properly display DCI content withoutartifacts (e.g., dark noise, dark contouring etc.) the EDR projectorshould have a “DCI Mode” that provides an experience that is identical,or at least similar to the standard DCI projector. It may be desirableto provide an improved experience, but it must be without anyobjectionable artifacts, and be in line with the artistic intent of thecontent creator.

To give some idea of the need to match performance between DCI and EDRprojectors, Table 1 below is the Reference Chart of Image Parameters andTolerances, as published in the “Digital Cinema System Specification”,Version 1.2 published Mar. 7, 2008:

TABLE 1 DCI IMAGE PARAMETER AND TOLERANCES Nominal Tolerances (Projected(Review Tolerances Image Parameters Image) Rooms) (Theatrical) PixelCount 2048 × 1080 or N/A N/A 4096 × 2160 Luminance Uniformity, 85% 80%to 90% 70% to 90% corners and sides of center of center of centerCalibrated White Luminance, 48 cd/m² ± 2.4 cd/m² ± 10.2 cd/m² center (14fL) (±0.7 fL) (±3.0 fL) Calibrated White Chromaticity, x = .3140, ± .002x, y ±. 006 x, y center from code values y = .3510 [3794 3960 3890]Color Uniformity of White Field, Matches center ±.008 x, y ±.010 x, ycorners Relative to Relative to center center Sequential Contrast 2000:11500:1 1200:1 minimum minimum minimum Intra-frame Contrast  150:1 100:1100:1 minimum minimum minimum Grayscale Tracking No visible color Novisible color No visible color shading shading shading ContouringContinuous, (same) (same) smooth ramp, with no visible steps TransferFunction Gamma 2.6 ± 2% 10 ± 5% 11 Per component Per component ColorGamut Minimum Color (same) (same) Gamut enclosed by white point, blackpoint11 and Red: 0.680 x, 0.320 y, 10.1 Y Green: 0.265 x, 0.690 y, 34.6Y Blue: 0.150 x, 0.060 y, 3.31 Y Color Accuracy Colorimetric +/− 4 deltaE12 +/− 4 delta E12 Match

By contrast, EDR projector systems have larger color spaces and dynamicranges than that specified above for a (perhaps minimally) compatibleDCI projector.

In many embodiments disclosed herein, novel dual modulator, triplemodulator and other multi-modulator projection display systems andtechniques may be employed for inputting DCI content into such EDRprojectors and converting such content and rendering video and/or imagedata—at or beyond the DCI specifications.

Many embodiments employ a combination of a MEMS array as a first (orearly) stage modulator that projects an intermediate illumination onto asecond (or later) stage DMD modulator.

EDR Dual-Modulator Projector Embodiments

EDR projector systems and dual modulation projector systems have beendescribed in commonly-owned patents and patent applications, including:

(1) U.S. Pat. No. 8,125,702 to Ward et al., issued on Feb. 28, 2012 andentitled “SERIAL MODULATION DISPLAY HAVING BINARY LIGHT MODULATIONSTAGE”;

(2) United States Patent Application 20130148037 to Whitehead et al.,published on Jun. 13, 2013 and entitled “PROJECTION DISPLAYS”

(3) United States Patent Application 20130147777 to Lau et al.,published on Jun. 13, 2013 and entitled “APPLICATION OF MEMS PIXELS INDISPLAY AND IMAGING DEVICES”; and

(4) United States Patent Application 20120038693 to Kang et al.,published on Feb. 16, 2012 and entitled “HIGH DYNAMIC RANGE PROJECTIONSYSTEM”.

-   -   which are hereby incorporated by reference in their entirety.

Many dual, triple, more than 2-modulation (all of which are hereinafterreferred to as “multi-modulation”) display systems are disclosed hereinuse beam steering to put light on the modulation chips only whereneeded.

FIG. 1 is one embodiment of a dual modulating projector display system100, comprising two or more digital projectors (as modulators). FIG. 1shows a monochrome display 100 according to this example embodiment.Display 100 comprises a light source 102. Light 104 from light source102 illuminates a first light modulator 106. Light source 102 maycomprise, for example: a laser; a xenon lamp; an array of lasers (e.g.,diodes or otherwise) or other solid-state light emitters; an arc lamp;or the like.

In one embodiment, the first light modulator 106 may comprise aplurality of controllable elements 106 a—e.g., on a fast switchingdevices, such as a MEMS device or the like. As will be described ingreater detail below (and in reference to FIGS. 2A-B and FIGS. 3A-C),elements 106 a may be selected such that they may be steered to reflectlight to a second modulator 110 by a suitable control circuit/controller116. The controller 116 may comprise a processor, a memory incommunication with the processor and such that the memory may compriseinstructions such that the controller may suitably control firstmodulator and second modulator (and other modulators, if they are in thesystem at issue) to perform the highlighting techniques as describedherein.

The set of controllable elements may also comprises a set ofcontrollable analog mirrors—possibly with switching speeds sufficientlyresponsive to provide subframe rendering for processing highlights asdescribed herein. In one embodiment, the switching response time ofelements 106 a may be fast enough—so as to reflect light onto the secondmodulator several times in a given frame of image data. For example,elements 106 a may affect a half frame, third frame, a quarter frame, or1/n frame illumination onto second modulator 110, as desired.

Light from first modulator 106 may pass through an optical system108—which may comprise sufficient optical components to perform adesired point spread function (PSF) of illumination onto secondmodulator 110. Depending on the ratio of elements 106 a in firstmodulator 106 to elements 110 a in second modulator 110, the desired PSFmay vary accordingly. For example, if the first modulator 106 is a MEMSarray and second modulator 110 is a DMD array, a typical MEMS array hasmany less elements 106 a (e.g., range from a few hundred to a fewthousand mirror elements, 100 to 2-3K)—than a DMD array that may be afew million mirror elements thereon (e.g. over 500K mirrors and over).

Second light modulator 110 may be controlled by control circuit 116 (asfirst light modulator 106 may be) and comprise a plurality ofcontrollable elements 110 a. Each controllable element 110 a can becontrolled to select a proportion of the light that is incident on theelement 110 a from first spatial light modulator 106 that is transmittedto a viewing area 114 (through, possibly a second optical system 112).

In some embodiments, second spatial light modulator 110 comprisesoptical reflective or transmissive elements 110 a that can be switchedbetween ON and OFF states, e.g., a DMD device. In such embodiments,second spatial light modulator 110 may be controlled by a controllerthat sets its elements to be ON or OFF.

Transfer optics 108 carries light from first light modulator 106 tosecond light modulator 110. This light is capable of illuminating theentire active area of second light modulator 110 when all elements 106 aof first spatial light modulator 106 are ON. This light could spreadpast the edges of second spatial light modulator 110. Transfer optics108 may blur the light. Transfer optics 108 may be characterized by atransfer function which at least approximates how light issuing from apoint on first spatial light modulator 106 will be spread over secondspatial light modulator 110. The pattern of light incident on secondlight modulator 110 can be estimated or determined from theconfiguration of first modulator 106 (i.e. from which elements 106 a areON and which elements 106 a are OFF) and the transfer function. Asuitable projection lens 112 focuses light from second spatial lightmodulator 110 onto a screen 114 for viewing. Screen 114 may comprise afront-projection screen or a rear-projection screen.

Although the embodiment of FIG. 1 depicts a single light channel, itwill be appreciated that the first and second modulators may bereplicated for each of a series of color channels within the projectorsuch that each color channel includes 2 optically offset reflectivemodulators. The series of color channels may comprise a red channel, agreen channel, and a blue channel. The light source may comprise, forexample, a plurality of colored laser light sources. In one embodiment,the light sources may be modulated either globally (in brightness)and/or spatially (locally) dimmed according to signals (not shown) froma controller (e.g., 116).

The intermediate signals to the second modulator may be, for example,based on a light field simulation comprising a point spread function oflight reflected by the first modulator and the offset. For example, theintermediate signals to the second modulator may be based on a pointspread function of light reflected by the first modulator in eachchannel and the offset in each channel. The offset in the channels maybe the same, or the offset of at least two channels is different and theintermediate signals to second modulator in each channel is based on atleast one of the offset and differences in offset between channels.

Another Multi-Modulator Embodiment

FIG. 2 is another embodiment of a dual modulating projector displaysystem 200 suitable for the purposes of the present application. Display200 may comprise a light source 202—which may comprise one light source(e.g. lamp or the like) or a plurality of point sources of light (e.g.,lasers, LEDs or the like). In the context of a digital movie projector,light source 202 in FIG. 2 may comprise one or more banks of laser lightsources (e.g., 202-1, 202-2, 202-3; 202-1′, 202-2′, 202-3′—where theremay be a plurality of colored light sources that when combined mayrender a white light—e.g., red, green and blue).

Light from source 202 may be piped into optical stage 204—which maycomprise a combiner 204-1 to combine the light from the RGB lasersources and integrating rod 204-2 which may improve the uniformity ofthe light. Light 203 may thereafter be transmitted through a diffuser206 to provide angular diversity to the light. Firstmodulator/pre-modulator 208 may input this light and—under control ofcontroller 220—may provide pre-modulator image processing, as describedfurther herein.

In one embodiment (and as shown in FIG. 2 ), first, pre-modulator 208may be a DMD array that—through a set of optical elements may processseparate color channels (e.g., 208-1, 208-2 and 208-3 for, e.g., red,green and blue channels). For merely exemplary purposes, pre-modulator208 may be a 1.2″, 2K mirror DMD, using standard prism design.Pre-modulator 208 may be controlled to display a binary half-toneimage—e.g., where the pixels are full ON or OFF (where light in the OFFstate may be dumped to offstate light 205). In other embodiments, analogMicro-Electro Mechanical System (MEMS) and/or other analog and/ordigital reflectors may be suitably controlled to redistribute light toform a different type of image.

In one embodiment, the pre-modulator/halftone DMD may spatially modulatethe uniform light field to produce a halftone image; —e.g., in which allpixels are either ON or OFF for the entire frame time or a portionthereof. The resulting halftone image—appropriately blurred—may producesufficient light levels on the primary modulator/pulse-width DMD,especially if bright-clipping is desired to be avoided. Since thepulse-width DMD may only reduce light levels, the blurred halftone imageshould be substantially everywhere greater than the desired screenimage—e.g., the input image. In some circumstances when an image featuresuch as a very bright dot on a black background may force theunavoidable condition of choosing either bright or dark-clipping, somebright-clipping may be intentionally allowed and the blurred halftoneimage would not be greater than the input, particularly the dot.

In one embodiment, to achieve low light levels and to avoid halos, theblurred halftone image may be set to be slightly greater than thedesired screen image. So, the blurred halftone image may substantiallybe a bandlimited minimum upper-bound on the desired screen image—e.g.,with the bandwidth limited by the optical blur. One embodiment (asfollows) tends to produce a bandlimited upper-bound on an image. It maynot be a minimum upper-bound, but may have a similar performance. Thisrelaxation may be desirable as a true minimum may be harder to achieve,although possible. In this embodiment, it may suffice that the “nobright clipping” property is substantially preserved.

In this embodiment, the halftone image may be formed by using a spatialdither pattern. The dither pattern may be defined over a rectangularblock of pixels and may be repeated over the entire image frame bytiling the pattern. The size of the pattern may be related to the sizeof the blur kernel, since the kernel smoothes the pattern. The size ofthe kernel may also determine the minimum non-zero light level—e.g., onepixel of the dither pattern turned ON and all others OFF may produce theminimum level. Further information on this projector system embodimentis disclosed in co-owned U.S. Patent Application No. 61/866,704 andentitled “SYSTEMS AND METHODS FOR LIGHT FIELD MODELING TECHNIQUES FORMULTI-MODULATION DISPLAYS”, filed on 16 Aug. 2013—and which is herebyincorporated by reference in its entirety.

This half tone image 207 may be transmitted through a Point SpreadFunction (PSF) optic stage 212. PSF optical stage may comprise manydifferent optical elements 210, 214 or the like—e.g., lenses, diffusers,reflectors or the like. It will suffice for the purposes of the presentapplication that PSF optic stage receives the half-tone image from thepre-modulator 208 and provide a desired defocusing of the half-toneimage (209) to the second modulator/prime modulator 216. As with firstmodulator 208, second modulator may be a DMD array that—through a set ofoptical elements may process separate color channels (e.g., 216-1, 216-2and 216-3 for, e.g., red, green and blue channels). For merely anotherexemplary purposes, pre-modulator 208 may be a 1.4″, 4K mirror DMD,using standard prism design.

Prime modulator 216 may receive light 209 and may be controlled bycontroller 220. Controller 220 may employ a light field simulation thatestimates and/or models the combined effect of half-toning and PSF todetermine local brightness on the prime modulator 216 on apixel-by-pixel basis. In other embodiments, such as those employing MEMSreflectors, controller 220 may similarly model the light fieldformation. From this model, controller 220 may calculate, estimate orotherwise determine the pixel values for the prime modulator 216 tomodify the light field to produce the final projected/rendered image.Light 213 may thereafter be transmitted through projections optics 218to form a final projected/rendered image on a projector screen (notshown). OFF light may be dumped to offstate light 211.

In many embodiments, a final image may be produced that is the productof the defocused half-tone image and the prime modulator image. In sucha final image, contrast may be in the range of 200,000:1.

As mentioned, in operation, the projector system of FIG. 2 may operatein various fashions: e.g., as a time-division or pulse-width modulator,operating two or more DMDs and/or reflectors in series—both acting aspulse-width modulators. Such operation tends to require precisetime-division alignment and pixel-to-pixel correspondence oftime-division sequencing.

As such the projector display system of FIG. 2 , may employ the firstDMD/reflector 208 as a “pre-modulator” or “premod modulator” and mayspatially modulate a light source by means of a halftone image that maybe maintained for a desired period of time (e.g., a frame or a portionthereof). This halftone image may be blurred to create aspatially-reduced-bandwidth light field that may be applied to thesecond DMD/reflector 216. The second DMD/reflector—referred to as theprimary modulator—may pulse-width modulate the blurred light field. Thisarrangement may tend to avoid both requirements mentioned above—e.g.,the precise time-division alignment and/or the pixel-to-pixelcorrespondence. In some embodiments, the two or more DMDs/reflectors maybe frame-aligned in time, and approximately spatially frame-aligned. Insome embodiments, the blurred light field from the premod DMD/reflectormay substantially overlap the primary DMD/reflector. In otherembodiments, the spatial alignment may be known and accounted for—e.g.,to aid in image rendering performance.

In one embodiment, the projector system may create a binary halftoneimage, which may be smoothed by optical components to create a reducedbandwidth version of the desired display image. The shape of the opticalcomponent PSF may determine the properties of the smoothing function.The shape of the PSF may influence display performance and thecomputational requirements of the system. In many embodiments, PSFshaping may have one or more of the following attributes and/or thefollowing guidelines:

(1) the PSF may smooth the sparest halftone pattern to a relatively flatfield. This may impose an approximate lower bound on the size of thePSF;

(2) larger PSFs may reduce the spatial frequency at which dualmodulation is active and may result in larger “halos” (as discussedfurther herein). This may require larger computational costs;

(3) the PSF may have limited bandwidth and limited rise-times. Higherbandwidth and rise-times may require greater compensation accuracy andlimit computational approximations;

(4) the PSF may be compact and the PSF spatial extent may be limited.The PSF may decay to zero. A slow decay, or strong PSF “tails”, maylimit image contrast and increase computational requirements;

(5) the PSF may be substantially radially symmetric. Any asymmetry maybe accounted for in the computation.

In one embodiment, the optically blurred PSF may substantially assumethe shape of a Gaussian, or a revolved raised-cosine function, or someother substantially radially symmetric peaked function with limitedspatial extent or the like. In many embodiments, the PSF should assumelimited spatial frequency, limited rise times and/or limited spatialextent. Spatial frequency and rise times may be usually correlated.Excessive spatial frequency or rise times may require denser samplingand greater modeling precision, increasing computational requirements.If the PSF varies over the image frame, a set of PSFs may be used, and aPSF interpolation method may be employed. PSFs with high spatialfrequencies that change with PSF position may require a denser model setfor proper interpolation, increasing computational requirements andcalibration complexity. It may not be desirable to have sharp spikes orridges on the PSF pulse. Also, it may be desirable the PSF shouldgradually decay at its perimeter rather than end abruptly there. Asmooth shape will have lower spatial frequencies and longer rise times.The spatial extent of the PSF may determine the size of computationoperators. PSFs with broad decaying “tails” may increase operator sizeand therefore computational requirements.

In merely one exemplary embodiment, the PSF represents the blur functionthat is applied to—e.g., a 5×5 dither pattern. So, the PSF may be largeenough to produce a relatively flat field from a halftone imagecomprising a 5×5 grid of ones, with all other halftone pixels zero. Ifthe blur function has a substantially Gaussian shape or the like, thenits diameter may range from 10 pixels to 20 pixels. In this example, alower and upper bound may be specified that limits the shape of the PSF.The lower bound may be a raised-cosine pulse and the upper bound may bea Gaussian pulse.

For merely one example, let LB be the lower bound and UB the upperbound. Let “r” be the distance from the center of the PSF, and N thesize of the side of the dither pattern, both in pixels. The pulseamplitude may then be normalized to the center value, as follows:LB(r)=0.9(½+½ cos(πr/N)) for r<NLB(r)=0 for r≥NUB(r)=1.1 exp(−(r/N){circumflex over ( )}2)

As may be noted, the lower bound decays to zero and the upper bounddecays as a Gaussian. The decay is significant to avoid the accumulationof too much light from PSF tails. It will be appreciated that many otherPSF shapes and functions are possible and that the scope of the presentapplication encompasses all such variations.

Referring attention to FIG. 3 , FIG. 3 depicts one embodiment of a highlevel flowchart 300 for operation of the optical processing/imageprocessing that may be affected with a dual, multi-modulator displaysystem, such as depicted in FIG. 1 . Uniform light 301 may be input intothe display system and a first modulator 302 (e.g., half-tone DMD orother modulator) provides a half-tone image to blurring optics 304.Thereafter, blurred image may be received by second modulator 306 (e.g.,pulse-width DMD or other modulator) further modulates the blurred imageto produce screen image 303. In one embodiment, flowchart 300 tracks aset of processor-readable instructions that may be stored in systemmemory in a controller. A controller may receive image data, produce ahalf-tone image (e.g. at 302), blur the half-tone image (e.g., at 304)and further modulate the image (e.g., at 306) to produce a final image.

In the context of the display system of FIG. 1 , the codewords to eachDMD device may be employed as two variables available to control theimages produced. In one embodiment, the blur function may be performedby an optic system and may be assumed to be constant for all images. Invarious display system designs, the design of the blur function and themethod of halftone encoding may be related and affect the performance ofthe display. In one such exemplary system, the followingobjectives/assumptions might be considered in order to determine asuitable choice of halftone encoding and blur function follow:

-   -   (1) No bright clipping: In one embodiment, the blurred light        field incident on the primary modulator/DMD may be everywhere        greater than the input image, the desired screen image. The        primary modulator/DMD may attenuate the light field.    -   (2) Small halo: Halos are dark clipping around a bright object        on a dark background. Given that a small bright object on a        black background is not bright clipped, the blurred light field        at the bright object may be greater than the bright object.        Because the light field may have reduced spatial bandwidth, it        may not be dark very near the bright object. The level of the        light field close to the bright object may be reduced by the        primary DMD as much as possible, but might still be greater than        the desired screen level, causing dark clipping. In some cases,        the dark clipping may represent elevated levels above true        black, often exhibiting loss of dark detail and loss of        contrast. Halos are the primary visual artifact caused by dark        clipping, but dark clipping may occur in any local region where        the blurred light field cannot represent a high-contrast, high        frequency pattern. The spatial extent of a local region may be        determined by the bandwidth of the light field which may be        determined by the size of the blur kernel or PSF.    -   (3) Adequate local contrast: The bandwidth of the light field        may be determined by the size of the blur PSF. A smaller PSF        allows for a higher bandwidth light field. But a smaller PSF        must be paired with a denser halftone pattern. A denser halftone        pattern may be associated with a smaller dither pattern size; it        may have fewer discrete levels and a higher first non-zero        level.    -   (4) The above objectives may compete. As such, many variations        and/or embodiments may be possible and/or desirable. This will        be discussed further herein with respect to DMD contrast ratios,        PSF size and local contrast considerations.

Table 1 below shows one exemplary 10×10 pattern showing the levelindexes. For a given level index, the numbered pixel and all lessernumbered pixels are turned ON while all greater numbered pixels are OFF.When a given level pattern is blurred, the result tends not to be flatand the modulated field may have some minimum. Table 2 below shows thenormalized minimum light levels for the each level index of Table1—showing the light level for the previous index. It will be appreciatedthat other pattern sizes and other spatial dithering patterns arepossible and are encompassed by the present application.

TABLE 1 Spatial Dither Pattern Exemplary 1 93 17 69 33 4 96 20 72 36 6137 77 29 89 64 40 80 32 92 13 85 5 45 53 16 88 8 48 56 73 25 65 97 21 7628 68 100 24 49 41 57 9 81 52 44 60 12 84 3 95 19 71 35 2 94 18 70 34 6339 79 31 91 62 38 78 30 90 15 87 7 47 55 14 86 6 46 54 75 27 67 99 23 7426 66 98 22 51 43 59 11 83 50 42 58 10 82

TABLE 2 Normalized Minimum Light Levels for Table 1 Pattern 0       0.919914 0.159947 0.679884 0.319955 0.029713 0.949726 0.189717 0.7096710.349671 0.599925 0.35989  0.759885 0.279915 0.879896 0.62969  0.3896510.789665 0.309641 0.909666 0.119936 0.839911 0.039901 0.439936 0.5199230.149697 0.869623 0.069613 0.469654 0.549684 0.719902 0.239875 0.6399140.959919 0.199903 0.749686 0.269715 0.669626 0.989631 0.229615 0.4799130.399908 0.559908 0.079919 0.799928 0.509626 0.429714 0.589671 0.1096780.829687 0.019936 0.939929 0.179932 0.699874 0.339925 0.009668 0.9296860.169673 0.689643 0.329627 0.619936 0.379903 0.7799   0.299927 0.8998860.609645 0.369611 0.769624 0.289596 0.889622 0.139927 0.859917 0.05989 0.459925 0.539903 0.129655 0.849579 0.049569 0.449609 0.529644 0.73988 0.25989  0.65992  0.979956 0.21991  0.729642 0.249675 0.649582 0.9695870.209571 0.499944 0.419922 0.57992  0.099932 0.819934 0.489581 0.4096740.569627 0.089634 0.809642

In this embodiment, for any particular input pixel, the level of thecorresponding pixel of the blurred halftone image should be greater. Toachieve the desired greater level at that pixel, all nearby pixels ofthe input image within the spatial extent of the blur kernel may beevaluated—e.g., any of those nearby pixels with a level less than thedesired level may be turned ON. One embodiment of this method may beaffected as follows:

For any particular input pixel, choose a level index such that the levelof the full-frame light field is greater than the pixel level. Forexample, it is possible to choose a level index—e.g., for the entireframe—that creates a halftone pattern that, when blurred, exceeds thepixel level.

Given this full-frame halftone pattern, all pixels whose PSFs do notcontribute light to the particular pixel may be turned OFF withoutaffecting the level at the particular pixel.

It should be noted that this method may not produce halftone tiles withparticular levels, giving the halftone image a blocky appearance.Rather, individual pixels may be turned ON or OFF, depending on theirindex and proximity to image features. In other embodiments, it may bepossible to affect error diffusion and/or local blue noise—e.g., wherethe halftone grid may be locally thresholded by the corresponding pixel.

It should be appreciated that while one embodiment may be affected by anordered dithering, it may be linked with a dilation to achieve anupper-bound. Smoothness may be a concern at the lowest levels—e.g., suchas only one pixel on for the dither pattern. It may be possible to applyother approaches, such as blue noise and/or FM-dithering, for differentsmoothness effects. For another example, consider a small bright objectat less than full brightness on black background. In this case, the halointroduced may be wider than desired. The dilation area may not be fullypopulated with ones. A more compact area with all ones may show less ofa halo because halo width of the display is greater than eye glarewidth. Reducing the brightness of small bright objects may reduce halowidth, rather than just reduce halo brightness.

FIG. 4 shows one embodiment of a method for producing a suitable binaryhalftone image. Halftone image module 400 may receive input image data401 and may dilate the image data to the extent of the blur kernel at402—to produce x(m,n), the dilated input image. The resulting binaryhalf-tone image, b(m,n), may be set to b(m,n)=1, ifx(m,n)>htLevel(m,n)—where htLevel(m,n) may be given as the values inTable 2 as the map—e.g., tiled over the entire image frame. The binaryhalf-tone image may be returned as b(m,n).

In one embodiment, the dilation operator may be employed to achieveclose to the min upper bound. Other embodiments may employ nonlinearfilters that may provide a max of elements under the kernel.

The primary modulator/pulse-width DMD modulates the blurred halftoneimage light field to produce the desired screen image. The Pulse-widthDMD can only attenuate light—so the light field may be an upper bound onthe desired screen image to prevent bright clipping. In addition, toprevent dark clipping, the light field may be a minimum upper bound. Theblurred halftone image light field may be computed, estimated orotherwise modeled using a model of the optical process. In oneembodiment, the optical process may be assumed to be only the blur—e.g.,the pre-modulator-to-primary modulator alignment may be ignored. In someembodiments, this may be the overall registration error.

In other embodiments, such alignment may be taken into consideration andproduce a correction factor to be applied. For example, on a realdisplay, the blurred light field frame created by the premod DMD may notbe perfectly aligned with primary DMD frame. For example, the lightfield image may be slightly rotated, shifted, or scaled to provideoverscan at frame edges. It may also be warped due to the blur optic andother optics. For such possibilities, a premod-to-primary map that mapspoints on the premod DMD to points on the primary DMD may be measuredand applied as a mapping—e.g., as a Look-Up Table (LUT) or the like.

FIG. 5 depicts one embodiment of a technique for generating apulse-width DMD compensation image, 503. Binary half-tone image (403,e.g., from FIG. 4 ) may be input into a blur model 502 and may bereciprocated at 504. The pulse-width DMD compensation image may bedetermined by dividing the input image by the modeled blurred halftoneimage light field—e.g. multiplying (at 506) the input image 501 by thereciprocal of the blurred half-tone image light field.

On the real display, the PSF shape for a given premod pixel may dependon its position on the premod frame. The blur optic may not blur allpremod positions the same. The PSF for pixels in a local area may beassumed to vary little and all pixels may be assumed to have the sameenergy (e.g., given a uniform light field incident on the premod).However, on a real display, each PSF may tend to be different. In oneembodiment, for a 2K frame, each PSF may be modeled separately, and/orapplied to a local portion of the image area—e.g., resulting in 2million PSFs that might be captured, stored, modeled, and usedcomputationally. Other embodiments may provide a simplifying model toreduce this complexity. Because the PSFs in a local region tend to besimilar, a single PSF model is used to represent all PSFs—e.g., at leastin local areas and/or local portions of the image area. Such,potentially localized, PSF models may be measured or otherwise modeledto provide suitable PSF models.

The primary DMD compensates the blurred light field to produce a finalscreen image. In one embodiment, light field compensation may beperformed on the primary DMD pixel grid. For this compensation process,the blurred light field may be represented on the primary pixel grid.However, the light field is formed by blurring the halftone image whichis on the premod pixel grid. In addition, the premod and primarymodulators may not be aligned.

To affect a suitable compensation process, there are two possiblealternative embodiments from which to choose. A first embodiment mightbe to model the light field on the premod grid and then map it to theprimary grid. A second embodiment might be to model the light field onthe primary grid by modeling the PSFs associated with each premod pixelon the primary grid. While the present application encompasses bothalternative embodiments, it will now be described the firstembodiment—i.e., to modeling the light field on the premod grid and mapit to the primary grid. In one embodiment, it may be possible to mappoints on the primary grid accounting for geometric and/or opticdistortions.

This first embodiment may be selected for the following reasons:

-   -   (1) because the PSF may substantially maintain its shape in a        local area. Thus, the light field may be modeled in the local        area on the premod by a standard convolution process on the        halftone image using a single PSF for the entire area.    -   (2) because of the premod-to-primary misalignment, the PSFs in a        local area on the primary may have different sample phases and        these may need to be accounted for. In some embodiments, because        the premod and primary may not be aligned, there may be some        sample-phase shift moving the inherently premod-grid-aligned PSF        models to the primary grid.    -   (3) If modeled on the primary, even though the PSF shape does        not change in a local area, different PSFs may need to be used        when computing the convolution because of the sample phase        change.    -   (4) The PSFs are inherently premod referenced. More PSFs would        need to be modeled and recorded for the primary than for the        premod.    -   (5) Modeling the light field may tend to be computationally        costly. For a practical implementation, the PSFs may need to be        subsampled and approximated. This might be simpler if performed        on the premod grid.    -   (6) Mapping the modeled light field from the premod to the        primary has a computational cost, but it may be less than the        cost of modeling the light field on the primary.    -   (7) Modeling the light field on the premod may be affected by        mapping the modeled light field from the premod to the primary.        It may be desirable that this map be accurate. For a particular        display, the premod-to-primary alignment is fixed. If the map        has errors, they may be fixed. For example, the errors may be        accounted for by modifications to the PSFs during the        calibration process—e.g., an offset error in the map may be        countered by an offset in the PSF model at that frame position.

In addition, because of premod-to-primary misalignment, the input imagemay be mapped to the premod grid for the process of computing thehalftone image. This process may not require as much accuracy as mappingthe light field to the primary. FIG. 6 depicts the afore-mentioned firstembodiment. The system—under controller direction—may receive inputimage 601. A primary-to-premod mapping 602 may be applied prior tocomputing the half tone image at 604. A light field model may be appliedat 606 and then a premod-to-primary map at 608 may be applied to producea primary registered light field 603. In some embodiments, the primarymay be one resolution (e.g., 4K) and the premod may be anotherresolution (e.g., 2K); but processing may be able to affect othermappings. For example, the processing may affect 2K/2K mapping—but the2K primary may be upconverted to 4K by the projector system. Othermappings are, of course, possible

DCI Content Mapping for EDR Projection Embodiments

Having now discussed several embodiments of EDR projector systems, itwill now be discussed how such EDR projector systems may handle an inputdata stream that may comprise different image data formats. Thus,however a given EDR projector system may be architected (e.g. as in theembodiments of FIG. 1 , FIG. 2 or otherwise), many embodiments of thepresent application are able to input image/video data, which maycomprise both EDR image/video data and/or DCI-specified images and/orvideo content.

As noted, EDR projection systems have the potential to significantlyimprove rendering performance—as compared to a standard DCI (DigitalCinema Initiative) projector. Display of standard DCI content istypically designed and graded for projectors that meet the DCI projectorspecification. These projectors have a more limited dynamic range andcolor space than the EDR projector. For example, even though DCIspecifies a minimum 2000:1 sequential contrast ratio for the projector,in practice few projectors exceed this value and thus all grading isusually done with an approximate contrast ratio of 2000:1.

In order to properly display DCI content without artifacts (e.g., darknoise, dark contouring etc.) the EDR projector should have a “DCI Mode”that provides an experience that is identical, or at least similar tothe standard DCI projector. It may be desirable to provide an improvedexperience, but it must be without any objectionable artifacts, and bein line with the artistic intent of the content creator.

FIG. 7 depicts one embodiment of EDR projector system 700 that furthercomprises image and/or video processing modules that may effectivelyrender both EDR and DCI image and/or video data 701 as input—that mayalso provide a DCI mode, as mentioned above. Projector 750 broadlycomprises light source 752 and dual modulators 754, 756 that transmitlight through projector optics 758 and onto screen 760. Projector 750may be architected in the fashion of FIGS. 1 and 2 described above—or inany manner that may affect rendering of EDR images/video.

FIG. 7 also depicts one embodiment of a video/image pipeline 720. At702, the pipeline accepts input data and/or metadata 701 and may detectwhether the present input is EDR, DCI or any other data format. Thesystem may determine if the input image data is DCI format, EDR formatand/or any other format in a number of ways. One possible way is todetect any metadata that is associated with the input image data.Metadata and/or tags may inform the system with which format the imagedata is compliant. Another possible way is for the controller/system toassay the image data and detect whether the image data is within DCIparameters (e.g., color gamut, luminosity, etc.). The controller/systemmay also detect any possible change in input image data parameters toautomatically detect image formats.

If pipeline is currently receiving DCI compliant image/video data at702, then pipeline 720 may perform all or any of the followingprocessing modules, in order to properly render the final image ontoscreen 760. Module 704 may perform a desired gamut mapping and/or colorspace conversion (e.g., via a 3×3 matrix—or any other known techniquesof gamut mapping). In one embodiment, module 704 may employ metadata todetermine the color space of the image data. Alternatively, module 704may determine color space by data testing. As will be discussed ingreater detail below, the color space conversion may be desirable and,in large measure, guided by the architecture of the projector system.For example, if the projector system is illuminated by a bank of lasersources, then the gamut mapping may help to improve the viewer'sexperience and perception of the final image.

Module 706 may provide a variety of dynamic range mapping techniques,depending on the architecture of the projector system—as well as desiredvisual effects. For example, in one embodiment, module 706 may determinethe desired performance based upon the metadata. Such performance may bea limitation of the process used to generate the image data. Module 708may effect artifact reduction that might arise from renderingDCI-specified content on an EDR projector system, as will be discussedfurther herein. Module 710 may affect a number of techniques that aredirected at the visual environment conditions in the viewing area. Theseenvironmental conditions may include room ambient data, (e.g., theambient light level and ambient color)—as well as room reflectance data(e.g., reflectance color and magnitude—as light from the screen may bereflected by the walls or other objects back to the screen).

It will be appreciated that module 710 may be employed in projectedsystems, other than dual or multi-modulator display system (e.g., highperformance and/or high f-number display systems, or any other systemswhich may exceed DCI specifications). For example, a 4K highlycollimated laser projection system (e.g., large format digitalequivalent) would also suffice. At 712, controller 720 may send controland/or data signals to the EDR projector in order to render the finalimage on screen 760. For example, controller may control and/or energizeany suitable display, e.g., an EDR dual modulation display or other highperformance display system.

It will be appreciated that different light sources may apply to variousembodiments of projector systems. For example, in various embodiments,light source 751 may comprise any suitable light source—e.g., Xenonlamp, lasers, LEDs, nanotube-based light sources. In addition, the lightsource may exhibit a native polarization, could be based on phosphors orother light emitting materials or light converting materials such asquantum dots.

In many embodiments, the functionality of Module 710 may be configuredto operate on separate images that are later combined. Such combinationsmay be electronic (e.g., image data product) or projected in a manner toform a complete image. The images may be, for example, left and rightchannels of a 3D image, or high and low resolution and/or spatialfrequency images. Typically, the images are of a same content subjectmatter of the same frame. For example, left and right view images of thesame scene in a 3D image, low and high spatial frequency images of asame frame (same basic picture or image being reproduced, one of theimages being composed of mainly low spatial frequency content, the otherbeing composed mainly of high spatial frequency content). In oneembodiment, the invention prepares low and high spatial frequencycontent images for each of left and right channels of a 3D image.

Processing of the images in module 710 may include additional hardwareor processing capability including, for example, adownsampler/upsampler, 3D converter, spatial frequency divider/converteror other hardware necessary to produce the different images. Metadataprovided with original image data may also be utilized to produce theimages or to direct other processing element in producing the differentimages. The metadata or production of additional images may be producedin a post-production process using, for example, professional displaymonitor. Alternatively the images may be provided in the original imagedata—such images may be captured at the source (e.g., different viewcameras/capture devices for 3D, and/or low and high spatial frequencycameras capturing a same scene).

The various processes within module 710 may operate individually on theadditional images and provide an output. For example, color spaceconversion may be separately performed and with different parameters foreach of the images (e.g., a first set of parameters to process lowspatial frequency images and a second set of parameters to process ahigh spatial frequency image). Once processed the images are output fordisplay. The images may be combined electronically (as a combined set ofimage data) and displayed or projected for viewing by a viewer.

Alternatively, the additional images may be displayed or projectedsimultaneously and integrated together when projected and/or viewed. Forexample, the images may be provided to separate projection systems, afirst projection system projecting a low spatial frequency image and asecond projection system projecting a high spatial frequency version ofthe same image. The projections may be simultaneous. The projectionsystems may be inversely synchronized such that they alternate betweenprojecting low spatial frequency content images and high spatialfrequency content images. In one embodiment, quad projectors areutilized and may be each configured to project images of any ofdifferent resolutions, spatial frequencies, and/or views (e.g., channelsof a 3D image) in any pattern. In one embodiment, quad projectors areutilized to project 3D images the left and right channels beingalternated between low and high spatial frequencies and projected fromdifferent projectors at different times.

Dynamic Range Processing Embodiments

The EDR projector has a vastly larger dynamic range (100X-1000X) than astandard DCI projector. In addition, an EDR projector may besignificantly more luminant than a standard DCI projector—108 nits forEDR projection, compared with 48 nits for DCI projection. Moreover, witha dual modulation DMD-DMD projector (e.g., as in FIGS. 1 and 2 ), bothmodulators may be used together—and in varying ways—to create an EDRimage.

FIG. 8 is one embodiment of a method (800) that may be employed by anEDR dual modulator projector system to handle input data that isDCI-compliant. In one embodiment, this method may be implemented insoftware, firmware or a combination in the controller of the projectorsystem. At 802, the controller may receive as input a combination ofEDR, DCI image/video data—or image/video data of any known format. At804, the controller and/or system may detect which format the inputimage data represents. In one case, if the input data is notDCI-compliant, then the system may process the input data according toEDR or other image format known.

If this input image data is DCI-compliant, then, at 808, thecontroller/system may set the first modulator (e.g. 106 in FIG. 1 or 208in FIG. 2 as a pre-modulator) to be fully turned ON. This would set theprojector system to its potentially maximum luminance. At 810, thecontroller/system may send control and/or data signals to the secondmodulator—so as to form the desired image being projected. In thismanner, the EDR projector may be used like a single modulator projector,which is sufficient to render DCI-compliant image data.

As this method for making a projector with similar contrast ratios to aDCI projector would be to leave the pre-modulator turned full on, theresult would be as a virtual single modulator system, and would tend tohave nearly identical dynamic characteristics to the DCI projector. Insome embodiments where the EDR projector can reproduce higher lightlevels, it is possible to adjust the output down to DCI levels, i.e., 48nits. Thus, suitable control signals may be made to either the lightsource (e.g., turn down the output of the light source) or—eitheralternatively or in combination—to set the first modulator to produce ahalftone less than fully ON. For example, if the EDR projector employslaser light source, maximum luminance may be in the range of 108 nits.Thus, it may be possible to set the spatial halftone substantially to aratio of 48/108—or (DCI Max Luminance)/(Max Luminance of the EDRsystem). It will be appreciated that other desired luminance ratios lessthan fully ON are contemplated by the present application and the scopeof the present application encompasses such other desired ratios. Inaddition, for other embodiments, It may be possible to consider othermaximum luminance projection (e.g., other than 108 nits) and otherembodiments are encompassed by the present application.

In operation, if the projector system received a combination of EDR andDCI image content, then the projector system may be able to switchbetween DCI-mode and EDR-mode, as appropriate. When EDR data is beinginput and rendered, then the projector system may employ the full rangeof dual modulator operation, as discussed above. When DCI data is beinginput and rendered, then the projector system may set the appropriate DClevel (e.g., fully ON or otherwise) of the first modulator and employthe second modulator as the image rendering modulator.

Another second method for a dynamic range processing method is to usethe dual modulation technique of the projector system, but to add alight floor below which the projector is never allowed to fall. Formerely one example, it may be possible to put the floor at 1/1800 of thepeak, which would result in the normal sequential contrast ratio of agood DCI projector. This method provides an image without artifacts, andwith some increase in the simultaneous contrast ratio. As before, it maybe possible to set the peak output to 48 Nits.

FIG. 9 is one embodiment of this second dynamic range processing method(900). As input image/video data is received at 902, the projectorsystem may determine at 904 whether the input image data is DCI content,EDR content or otherwise. If the content is EDR, then the projectorsystem may determine to process the EDR or other content data as desiredat 906. In the case that the input data is DCI content, the projectorsystem may adjust either image data signals, control signal to one orboth modulators, light source(s) (or all of the above) in order toaffect a light floor level of illumination.

To affect this light floor, there are many possible techniques touse—either singly or in combination. For one possible embodiment, theprojector system may alter the image data such that dark regions of theimage have their luminance boosted to the light floor level. In anotherpossible embodiment, the projector system may adjust the light ratio ofthe first modulator to be more than 48/108, as discussed above. Inaddition, there are other techniques that may be employed to ensure suchlight floor level. For example, it is possible to turn on the overheadlights in the auditorium to achieve this light floor level.

Extending this idea, the floor could be lowered below 1/1800 (that is,Contrast Ration (CR)=1800:1) to around 1/5000 (CR=5000:1) withoutencountering serious artifacts or serious changes to the artisticintent. For CRs between 5000:1 and 50000:1, artifacts may become visibleand the artistic effect/intent may tend to shift. Thus, for someembodiments, it may not be desired to have the CR greater than50000:1—as the artistic intent is likely to shift. Possible artifactsmay comprise be black clipping, noise and grading errors.Inconsistencies in the grading or any conversion and/or processing ofthe grading and/or graded image data may tend to become visible.

Yet another third method for dynamic range processing may be to use aninverse tone mapping technique that analyses the DCI image data anddynamically adjusts the picture based upon a mapping algorithm.Techniques as disclosed in the following co-owned application may beemployed to effect an inverse tone mapping and/or dynamic pictureadjustments:

-   -   (1) United States Patent Application 20130148029 to Gish et al.,        published on Jun. 13, 2013 and entitled “EXTENDING IMAGE DYNAMIC        RANGE”; and    -   (2) United States Patent Application 20130076763 to Messmer,        published on Mar. 28, 2013 and entitled “TONE AND GAMUT MAPPING        METHODS AND APPARATUS”.        -   all of which are incorporated by reference herein in their            entirety.

Color Space Processing Embodiments

The EDR projector has a larger color space than the DCI projector. FIG.10 is a color gamut map 1000, depicting the DCI-standard color gamut(also known as P3) 1006 within the context of the CIE 1931 chromaticitydiagram 1000—with 1002 being the spectral locus. In several embodimentsof the EDR projector system (e.g., FIG. 2 ), the light source maycomprise a single wavelength sources that may further comprises two ormore single wavelength sources that are relatively spaced closetogether.

For example, one embodiment may have a red laser light course 1004-r oralternatively, two red laser light sources (e.g., splitting 1004-r into1004-r 1 and 1004-r 2), one or two green laser light sources (e.g.,splitting 1004-g into 1004-g 1 and 1004-g 2) and one or two blue laserlight sources (e.g., splitting 1004-b into 1004-b 1 and 1004-b 2)—thatdefine the color gamut 1004 as shown embedded in the CIE 1931chromaticity diagram.

In various embodiments, it is possible to implement methods that map awider color space (e.g. 1004) to a narrower one (e.g., 1006). One suchmethod may be a 3×3 CSC (color space conversion) matrix. This could beused to map the native EDR color space into the DCI color space withhigh accuracy.

In another embodiment, it may be possible to expand the DCI (P3) contentinto the expanded EDR color space (e.g., 3D LUTs, gamut mapping). Inmany embodiments, it would be desirable to avoid creating undesirableartifacts (e.g., color shifts, excessive saturation) while performingcolor expansion. Several color space processing techniques may be foundin many of the above-incorporated by reference co-owned patentapplications.

Various functions described herein can be implemented in hardware,software, or any combination thereof. If implemented in software, thefunctions can be stored on or transmitted over as one or moreinstructions or code on a computer-readable medium. Computer-readablemedia includes computer-readable storage media. A computer-readablestorage media can be any available storage media that can be accessed bya computer. By way of example, and not limitation, suchcomputer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that can be used to carry or storedesired program code in the form of instructions or data structures andthat can be accessed by a computer. Disk and disc, as used herein,include compact disc (CD), laser disc, optical disc, digital versatiledisc (DVD), floppy disk, and Blu-ray disc (BD), where disks usuallyreproduce data magnetically and discs usually reproduce data opticallywith lasers. Further, a propagated signal is not included within thescope of computer-readable storage media. Computer-readable media alsoincludes communication media including any medium that facilitatestransfer of a computer program from one place to another. A connection,for instance, can be a communication medium. For example, if thesoftware is transmitted from a website, server, or other remote sourceusing a coaxial cable, fiber optic cable, twisted pair, digitalsubscriber line (DSL), or wireless technologies such as infrared, radio,and microwave, then the coaxial cable, fiber optic cable, twisted pair,DSL, or wireless technologies such as infrared, radio and microwave areincluded in the definition of communication medium. Combinations of theabove should also be included within the scope of computer-readablemedia.

Alternatively, or in addition, the functionally described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Program-specific Integrated Circuits (ASICs), Program-specificStandard Products (ASSPs), System-on-a-chip systems (SOCs), ComplexProgrammable Logic Devices (CPLDs), etc.

A detailed description of one or more embodiments of the invention, readalong with accompanying figures, that illustrate the principles of theinvention has now been given. It is to be appreciated that the inventionis described in connection with such embodiments, but the invention isnot limited to any embodiment. The scope of the invention is limitedonly by the claims and the invention encompasses numerous alternatives,modifications and equivalents. Numerous specific details have been setforth in this description in order to provide a thorough understandingof the invention. These details are provided for the purpose of exampleand the invention may be practiced according to the claims without someor all of these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

The invention claimed is:
 1. A device comprising: a first modulator; asecond modulator; and a controller configured to receive input imagedata comprising a plurality of image formats; determine whether theinput image data comprises Digital Cinema Initiative (DCI)-compliantimage data; determine whether the input image data comprises EnhancedDynamic Range (EDR) image data; processing the EDR image data inresponse to determining that the input image data comprises the EDRimage data; set a luminance of the first modulator as a function of aratio of a DCI maximum luminance to a maximum luminance of an EDRprojector system in response to processing the EDR image data anddetermining that the input image data comprises the DCI image data;perform dynamic range processing on the DCI image data by creating ablurred halftone image on the first modulator; and render the DCI imagedata that is processed from the first modulator on the second modulator.2. The device of claim 1, wherein, to determine whether the input imagedata comprises the DCI image data, the controller is further configuredto: detect metadata associated with the input image data, the metadataindicating that the input image data is DCI-compliant, or assay theinput image data to detect a plurality of image data parametersindicating that the input image data is DCI-compliant.
 3. The device ofclaim 1, wherein, to perform the dynamic range processing on the DCIimage data, the controller is further configured to: set the firstmodulator to a desired luminance level, and send signals to the secondmodulator to modulate light from the first modulator to form a desiredimage from the input image data.
 4. The device of claim 3, wherein, toset the first modulator to the desired luminance level, the controlleris further configured to: set the first modulator to transmit a fullluminosity.
 5. The device of claim 3, wherein, to set the firstmodulator to the desired luminance level, the controller is furtherconfigured to: set the first modulator to substantially transmit theratio of the DCI maximum luminance to the maximum luminance of the EDRprojector system.
 6. The device of claim 3, wherein, to send the signalsto the second modulator to form the desired image, the controller isfurther configured to: render the desired image with a desired minimumlevel of a full luminance output of the EDR projector system.
 7. Thedevice of claim 6, wherein the desired minimum level is substantially1/1800 of the maximum luminance of the EDR projector system.
 8. Thedevice of claim 1, wherein the controller is further configured to:gamut map DCI image data to a desired color gamut.
 9. The device ofclaim 1, wherein, to gamut map DCI image data to a desired color gamut,the controller is further configured to: expand the DCI image data to afull color gamut of the EDR projector system.
 10. A multi-modulationprojection display system comprising: a first modulator comprising aplurality of analog mirrors; a second modulator comprising a pluralityof mirrors; and a controller configured to: receive input image datacomprising at least one bright feature on a black background, determinewhether the input image data comprises Digital Cinema Initiative(DCI)-compliant image data, determine whether the input image datacomprises Enhanced Dynamic Range (EDR) image data, process the EDR imagedata in response to determining that the input image data comprises theEDR image data, set a luminance of the first modulator as a function ofa ratio of a DCI maximum luminance to a maximum luminance of an EDRprojector system in response to processing the EDR image data anddetermining that the input image data comprises the DCI image data,perform dynamic range processing on the DCI image data by creating ablurred halftone image on the first modulator, and render the DCI imagedata that is processed from the first modulator on the second modulator.11. The multi-modulation projection display system of claim 10, wherein,to determine whether the input image data comprises the DCI image data,the controller is further configured to: detect metadata associated withthe input image data, the metadata indicating that the input image datais DCI-compliant; or assay the input image data to detect a plurality ofimage data parameters indicating that the input image data isDCI-compliant.
 12. The multi-modulation projection display system ofclaim 10, wherein, to perform the dynamic range processing on the DCIimage data, the controller is further configured to: set the firstmodulator to a desired luminance level, and send signals to the secondmodulator to modulate light from the first modulator to form a desiredimage from the input image data.
 13. The multi-modulation projectiondisplay system of claim 12, wherein, to set the first modulator to thedesired luminance level, the controller is further configured to: setthe first modulator to transmit a full luminosity of themulti-modulation projection display system.
 14. The multi-modulationprojection display system of claim 12, wherein, to set the firstmodulator to the desired luminance level, the controller is furtherconfigured to: set the first modulator to substantially transmit theratio of the DCI maximum luminance to the maximum luminance of the EDRprojector system.
 15. The multi-modulation projection display system ofclaim 12, wherein, to send the signals to the second modulator to formthe desired image, the controller is further configured to: render thedesired image with a desired minimum level of a full luminance output ofthe EDR projector system.
 16. The multi-modulation projection displaysystem of claim 15, wherein the desired minimum level is substantially1/1800 of the maximum luminance of the EDR projector system.
 17. Themulti-modulation projection display system of claim 10, wherein thecontroller is further configured to: gamut map DCI image data to adesired color gamut.
 18. The multi-modulation projection display systemof claim 17, wherein, to gamut map the DCI image data to the desiredcolor gamut, the controller is further configured to: expand the DCIimage data to a full color gamut of the EDR projector system.